Behavioral Science: Enhancing Our Approach to the Development of Effective Additional Risk Minimization Strategies.

Additional risk minimization strategies may be required to assure a positive benefit-risk balance for some therapeutic products associated with serious adverse drug reactions/risks of use, without which these products may be otherwise unavailable to patients. The goals of risk minimization strategies are often fundamentally to influence the behavior of healthcare professionals (HCPs) and/or patients and can include appropriate patient selection, provision of education and counselling, appropriate medication use, adverse drug reaction monitoring, and adoption of other elements to assure safe use, such as pregnancy prevention. Current approaches to additional risk minimization strategy development rely heavily on information provision, without full consideration of the contextual factors and multi-level influences on patient and HCP behaviors that impact adoption and long-term adherence to these interventions. Application of evidence-based behavioral science methods are urgently needed to improve the quality and effectiveness of these strategies. Evidence from the fields of adherence, health promotion, and drug utilization research underscores the value and necessity for using established behavioral science frameworks and methods if we are to achieve clinical safety goals for patients. The current paper aims to enhance additional risk minimization strategy development and effectiveness by considering how a behavioral science approach can be applied, drawing from evidence in understanding of engagement with pharmaceutical medicines as well as wider public health interventions for patients and HCPs.

The Reporting Recommendations Intended for Pharmaceutical Risk Minimization Evaluation Studies: Standards for Reporting of Implementation Studies Extension (RIMES-SE).

INTRODUCTION: The Reporting recommendations Intended for pharmaceutical risk Minimization Evaluation Studies (RIMES) was developed to improve the quality of reporting of risk minimization program evaluations. In light of continued inadequacies in study reporting, and high-profile program implementation failures, we updated the RIMES Checklist to incorporate additional concepts from the Standards for Reporting of Implementation studies (StaRI). METHODS: The development of the updated checklist, the RIMES-StaRI Extension (RIMES-SE), entailed developing a study protocol and drafting an initial pool of items based on a mapping of the RIMES against the StaRI checklist. A modified e-Delphi exercise was then conducted to determine the importance and understandability of items for checklist inclusion. An expert workshop and an online commentary period for additional feedback followed. RESULTS: The RIMES-SE contains 27 items. It includes two signature features of the StaRI Checklist: 1) a dual strand of items (represented in two columns) describing the risk minimization program (the 'intervention') and the corresponding implementation strategy; and 2) applicable to an array of different research methodologies. CONCLUSIONS: The RIMES-SE Statement and Checklist extends the reporting guidelines set forth in the original RIMES Checklist via inclusion of key implementation science concepts. It is intended to improve the quality and transparency of reporting of risk minimization evaluation studies so as to advance drug safety science.

The Evolve to Next-Gen ACT Network: An evolving open-access, real-world data resource primed for real-world evidence research across the Clinical and Translational Science Award Consortium.

The ACT Network was funded by NIH to provide investigators from across the Clinical and Translational Science Award (CTSA) Consortium the ability to directly query national federated electronic health record (EHR) data for cohort discovery and feasibility assessment of multi-site studies. NIH refunded the program for expanded research application to become "Evolve to Next-Gen ACT" (ENACT). In parallel, the US Food and Drug Administration has been evaluating the use of real-world data (RWD), including EHR data, as sources of real-world evidence (RWE) for its regulatory decisions involving drug and biological products. Using insights from implementation science, six lessons learned from ACT for developing and sustaining RWD/RWE infrastructures and networks across the CTSA Consortium are presented in order to inform ENACT's development from the outset. Lessons include intentional institutional relationship management, end-user engagement, beta-testing, and customer-driven adaptation. The ENACT team is also conducting customer discovery interviews with CTSA hub and investigators using Innovation-Corps@NCATS (I-Corps™) methodology for biomedical entrepreneurs to uncover unmet RWD needs. Possible ENACT value proposition hypotheses are presented by stage of research. Developing evidence about methods for sustaining academically derived data infrastructures and support can advance the science of translation and support our nation's RWD/RWE research capacity.

Applying Customer Discovery Method to a Chronic Disease Self-Management Mobile App: Qualitative Study.

BACKGROUND: A significant health challenge is evident in the United States, with 6 in 10 adults having a chronic disease and 4 in 10 adults having 2 or more. Chronic disease self-management aims to prevent or delay disease progression and disability and reduce mortality risk. The evidence to support the use of information technology tools, including mobile apps, web-based portals, and web-based educational interventions, that support disease self-management and improve clinical outcomes is growing. Customer discovery and value proposition design methodology is a form of stakeholder engagement and is based on marketing and lean start-up business methods. As applied in health care, customer discovery and value proposition methodology can be used to understand the clinical problem and articulate the product's hypothesized unique value proposition relative to alternative options that are available to end users. OBJECTIVE: This study aims to describe the experience and findings of academic researchers applying the customer discovery and value proposition methodology to identify stakeholders, needs, adaptability, and sustainability of a chronic disease self-management mobile app (CDapp). The motivation of the work is to make mobile health app interventions accessible and acceptable for all segments of patients' chronic diseases. METHODS: Data were obtained through key informant interviews and analyzed using rapid qualitative analysis techniques. The value proposition framework was used to build the interview guide. The aim was to identify the needs, challenges (pains), and potential benefits (gains) of the CDapp for our stakeholders. RESULTS: Our results showed that the primary consumers (end users) of a CDapp were the patients. The app adopters (decision makers) can be medical center leaders including population health department managers or insurance providers, while the consumer adoption influencers (influencers or saboteurs) are clinicians and patient caregivers. We developed an ecosystem map to visualize the clinical practice workflow and how an app for chronic disease management might integrate within an academic health care center or system. A value proposition for the identified customer segments was generated. Each stakeholder segment was working within a different framework to improve patient self-management. Patients needed help to adhere to self-care activities and they needed tailored health education. Health care leaders aim to improve the quality of care while reducing costs and workload. Clinicians wanted to improve patient education and care while reducing the time burden. Our results also showed that within academic medical centers, there were variations regarding patients' self-reported abilities to manage their diseases. CONCLUSIONS: Customer discovery is a useful form of stakeholder engagement when designing studies that seek to implement, adapt, and sustain an intervention. The customer discovery and value proposition methodology can be used as an alternative or complementary approach to formative research to generate valuable information in a brief period.

Evolution of Cross-Sectional Survey Protocol Quality Over Time: A Case Series of Index U.S. REMS Knowledge Survey Protocols (2007-2020).

INTRODUCTION: Surveys are commonly used to assess effectiveness of FDA-required risk evaluation and mitigation strategies (REMS) for drugs and biologics in the United States. OBJECTIVE: The aim of this study was to assess the scientific rigor of REMS knowledge survey protocols submitted to FDA and compare protocols before and after FDA's 2012 public workshop and 2019 draft guidance. METHOD: A content analysis of index survey protocols submitted to FDA (2007-2020) for single-product REMS with elements to assure safe use (39 programs, 78 protocols) was conducted. Each protocol was scored against 52 core essential elements (CEE), abstracted from FDA's guidance and grouped into six domains: study objective (n = 5), study design (n = 18), survey instrument (n = 9), participant recruitment (n = 7), survey administration (n = 9), and statistical analysis plan (n = 4). Scores were collected by time periods: (A) Oct 2007 to Jul 2012; (B) Aug 2012 to Feb 2019; (C) Mar 2019 to Dec 2020; and compared using logistic generalized linear mixed models adjusting for domain, survey population, vendor, program, and protocol. RESULTS: There were 30 (38.5%), 40 (51.3%), and 8 (10.3%) protocols submitted in time period A, B, and C, respectively. Adjusted marginal means of elements present (on the probability scale) by time period were 0.5816 (SE = 0.0242), 0.6429 (SE = 0.0229), and 0.7543 (SE = 0.0394). The likelihood of missing a CEE declined over time (adjusted p-value = 0.0094, time period A vs C). The statistical analysis plan domain had the most improvement; study design remained the weakest domain with the scientific justification CEE particularly underrepresented. CONCLUSION: The rigor of REMS knowledge survey protocols improved over time consistent with FDA's efforts to advance regulatory science, but gaps remain.

Adaptation of a Mobile Interactive Obesity Treatment Approach for Early Severe Mental Illness: Protocol for a Mixed Methods Implementation and Pilot Randomized Controlled Trial.

BACKGROUND: Obesity is common in individuals with severe mental illness (SMI), contributing to a significantly shortened lifespan when compared to the general population. Available weight loss treatments have attenuated efficacy in this population, underscoring the importance of prevention and early intervention. OBJECTIVE: Here, we describe a type 1 hybrid study design for adapting and pilot-testing an existing mobile health intervention for obesity prevention in individuals with early SMI and Class I or early-stage obesity, defined as a BMI of 30-35. METHODS: An existing, evidence-based interactive obesity treatment approach using low-cost, semiautomated SMS text messaging was selected for adaptation. Community mental health clinics and Clubhouse settings in Eastern Missouri and South Florida were identified to participate. This study has the following 3 aims. First, using the Enhanced Framework for Reporting Adaptations and Modifications to Evidence-based interventions, contextual aspects of the clinical and digital treatment environments are identified for adaptation, considering 5 main stakeholder groups (clinical administrators, prescribing clinicians, case managers, nurses, and patients). Following a 2-week trial of unadapted SMS text messaging, Innovation Corps methods are used to discover needed intervention adaptations by stakeholder group and clinical setting. Second, adaptations to digital functionality and intervention content will be made based on themes identified in aim 1, followed by rapid usability testing with key stakeholders. A process for iterative treatment adaptation will be developed for making unplanned modifications during the aim 3 implementation pilot study. Individuals working in partner community mental health clinics and Clubhouse settings will be trained in intervention delivery. Third, in a randomized pilot and feasibility trial, adults with 5 years or less of treatment for an SMI diagnosis will be randomized 2:1 to 6 months of an adapted interactive obesity treatment approach or to an attentional control condition, followed by a 3-month extension phase of SMS text messages only. Changes in weight, BMI, and behavioral outcomes, as well as implementation challenges, will be evaluated at 6 and 9 months. RESULTS: Institutional review board approval for aims 1 and 2 was granted on August 12, 2018, with 72 focus group participants enrolled; institutional review board approval for aim 3 was granted on May 6, 2020. To date, 52 participants have been enrolled in the study protocol. CONCLUSIONS: In this type 1 hybrid study design, we apply an evidence-based treatment adaptation framework to plan, adapt, and feasibility test a mobile health intervention in real-world treatment settings. Resting at the intersection of community mental health treatment and physical health promotion, this study aims to advance the use of simple technology for obesity prevention in individuals with early-stage mental illness. TRIAL REGISTRATION: ClinicalTrials.gov NCT03980743; https://clinicaltrials.gov/ct2/show/NCT03980743. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/42114.

Mixed methods evaluation of the inaugural year of the Cancer Prevention and Control Research Network's (CPCRN) scholars program.

PURPOSE: A diverse workforce trained in dissemination & implementation (D&I) science is critical for improving cancer outcomes and reducing cancer-related health disparities. This study aims to describe and evaluate impact of the Cancer Prevention and Control Research Network (CPCRN) Scholars Program in preparing scholars for collaborative careers in cancer control and implementation research and practice, and offers evaluation-driven recommendations for program improvements. METHODS: The CPCRN Scholars Workgroup conducted a sequential, mixed methods evaluation. We collected baseline and follow-up surveys and invited all 20 scholars and ten mentors to participate in an exit interview. We assessed the experience with the Scholar's program, ratings of D&I competences, progress on their project, feedback about the curriculum, and understanding of implementation science. RESULTS: Over 86% partially or fully completed their project within 9 months; 78% of scholars engaged with a CPCRN workgroup. Scholars rated the following program components as valuable: the Putting Public Health Evidence in Action (PPHEIA) training (88.9%), D&I training modules (83.3%), and webinars (kickoff webinar-88.9% and selecting theories/models-88.9%). There was an increase in D&I competencies from baseline to posttest, with the greatest in community engagement topics. About 78% reported that they were satisfied with format of the activities and increased confidence in ability to discuss D&I concepts. From the qualitative interviews, the benefit of the program was becoming more knowledgeable about D&I research and networking. CONCLUSION: The inaugural year of the program yielded positive results, particularly related to increasing knowledge about D&I science and cancer control. This program builds the capacity of students, researchers and practitioners in D&I science.

Dissemination of an evidence-based behavioral intervention to alleviate distress in caregivers of children recently diagnosed with cancer: Bright IDEAS.

BACKGROUND: Four multisite randomized clinical trials of > 1400 caregivers of children newly diagnosed with cancer showed that the Bright IDEAS (BI) paradigm of problem-solving skills training is an acceptable and efficacious approach to alleviating the high levels of distress they experience. To facilitate providing evidence-based caregiver support as recommended in the pediatric oncology standards of care, the project described here was designed to disseminate BI to 200 psychosocial professionals. PROCEDURE: We partnered with the Children's Oncology Group (COG), Association of Pediatric Oncology Social Workers (APOSW), Association of Pediatric Hematology/Oncology Nurses (APHON), and special interest group in pediatric hematology/oncology of the Society for Pediatric Psychology (SPP). Membership surveys revealed substantial enthusiasm for training in BI. We structured training to include review of the evidence base for BI, role plays, and strategies for implementation at individual sites. Four conference calls designed to enhance implementation were held one, two, three, and five months after training. RESULTS: Ten 1.5-day workshops were held in conjunction with annual meetings of COG, APOSW, APHON, and SPP. A total of 209 psychosocial clinicians from 134 sites were trained. Evaluations were highly favorable. Trainees had provided BI to 545 individuals as of the last conference call. CONCLUSIONS: Initial dissemination goals were met. BI is now available at numerous pediatric oncology centers, but it has not become part of routine care. Future work focused on implementation might consider top-down approaches that include direct communication with pediatric oncologists and hospital leaders about the benefits of incorporating this evidence-based intervention systemically.

Using Innovation-Corps (I-Corps™) Methods to Adapt a Mobile Health (mHealth) Obesity Treatment for Community Mental Health Settings.

Background: We employed Innovation Corps (I-Corps™) methods to adaptation of a mobile health (mHealth) short-message-system (SMS) -based interactive obesity treatment approach (iOTA) for adults with severe mentall illness receiving care in community settings. Methods: We hypothesized "jobs to be done" in three broad stakeholder groups: "decision makers" (DM = state and community clinic administrators), "clinician consumers" (CC = case managers, peer supports, nurses, prescribers) and "service consumers" (SC = patients, peers and family members). Semistructured interviews (N = 29) were recorded and transcribed ver batim and coded based on pragmatic-variant grounded theory methods. Results: Four themes emerged across groups: education, inertia, resources and ownership. Sub-themes in education and ownership differed between DM and CC groups on implementation ownership, intersecting with professional development, suggesting the importance of training and supervision in scalability. Sub-themes in resources and intertia differed between CC and SC groups, suggesting illness severity and access to healthy food as major barriers to engagement, whereas the SC group identified the need for enhanced emotional support, in addition to pragmatic skills like menu planning and cooking, to promote health behavior change. Although SMS was percieved as a viable education and support tool, CC and DM groups had limited familiarity with use in clinical care delivery. Conclusions: Based on customer discovery, the characteristics of a minimum viable iOTA for implementation, scalability and sustainability include population- and context-specific adaptations to treatment content, interventionist training and delivery mechanism. Successful implementation of an SMS-based intervention will likely require micro-adaptations to fit specific clinical settings.

A flexible mixed-data model applied to claims data for post-market surveillance of prescription drug safety behavior.

We develop a new modeling framework for jointly modeling first prescription times and the presence of risk-mitigating behavior for prescription drugs using real-world data. We are interested in active surveillance of clinical quality improvement programs, especially for drugs which enter the market under an FDA-mandated Risk Evaluation and Mitigation Strategy (REMS). Our modeling framework attempts to jointly model two important aspects of prescribing, the time between a drug's initial marketing and a patient's first prescription of that drug, and the presence of risk-mitigating behavior at the first prescription. First prescription times can be flexibly modeled as a mixture of component distributions to accommodate different subpopulations and allow the proportion of prescriptions that exhibit risk-mitigating behavior to change for each component. Risk-mitigating behavior is defined in the context of each drug. We develop a joint model using a mixture of positive unimodal distributions to model first prescription times, and a logistic regression model conditioned on component membership to model the presence of risk-mitigating behavior. We apply our model to two recently approved extended release/long-acting (ER/LA) opioids, which have an FDA-approved blueprint for best prescribing practices to inform our definition of risk-mitigating behavior. We also apply our methods to simulated data to evaluate their performance under various conditions such as clustering.

Designing for Dissemination and Sustainability to Promote Equitable Impacts on Health.

Designing for dissemination and sustainability (D4DS) refers to principles and methods for enhancing the fit between a health program, policy, or practice and the context in which it is intended to be adopted. In this article we first summarize the historical context of D4DS and justify the need to shift traditional health research and dissemination practices. We present a diverse literature according to a D4DS organizing schema and describe a variety of dissemination products, design processes and outcomes, and approaches to messaging, packaging, and distribution. D4DS design processes include stakeholder engagement, participatory codesign, and context and situation analysis, and leverage methods and frameworks from dissemination and implementation science, marketing and business, communications and visualarts, and systems science. Finally, we present eight recommendations to adopt a D4DS paradigm, reflecting shifts in ways of thinking, skills and approaches, and infrastructure and systems for training and evaluation.

Core functions and forms of Bright IDEAS: A multi-methods evaluation of the adoption of an evidence-based psychosocial training program through iterative adaptation.

Background: Despite efforts to widely disseminate interventions designed to increase access to quality supportive care to pediatric cancer patients and their families, many of these interventions fail to meet expectations once deployed in real-life clinical settings. This study identifies the functions and forms of Bright IDEAS: Problem-Solving Skills Training, an evidence based psychosocial intervention for caregivers of children recently diagnosed with cancer, to identify pragmatic program adaptations in its real-world clinical implementation. We compare intervention adoption before and after adaptations to the Bright IDEAS training program as part of a national training program designed to disseminate the intervention. Methods: 209 pediatric psychosocial oncology practitioners representing 134 unique institutions were trained during 10 in-person 8-hour workshops (2015-2019). Functions and forms of Bright IDEAS were identified, and adaptations made to the training agenda and curriculum based on practitioner feedback following implementation in local institutions. Mixed method evaluation included longitudinal surveys at 6- and 12-months post training; and qualitative interviews among a subgroup of practitioners (N = 47) to understand and compare perspectives on intervention adoption and barriers to implementation before and after adaptations to the Bright IDEAS training program. The RE-AIM framework was used to guide dissemination evaluation. Results: A total of four adaptations were tailored to the identified forms of the intervention: case studies; pre-training reading materials; training videos; and letters of institutional support from primary supervisor. Pre- and post-training adaptations to the Bright IDEAS training program were mapped to RE-AIM constructs. Quantitative findings demonstrate that adaptations appeared to improve adoption and usage overall. Conclusion: This study provides insight into how contextual factors influence psychosocial practitioners' capacity to adopt, implement, and maintain Bright IDEAS in the clinical setting. This study demonstrates the use of real-time stakeholder feedback to guide intervention translation from research to practice settings.

The Effect of the Affordable Care Act on Women's Postpartum Insurance and Depression in 5 States That Did Not Expand Medicaid, 2012-2015.

BACKGROUND: Before the Affordable Care Act (ACA), most women who gained pregnancy-related Medicaid were not eligible for Medicaid as parents postpartum. The ACA aimed to expand health insurance coverage, in part, by expanding Medicaid; introducing mandates; reforming regulations; and establishing exchanges with federal subsidies. Federal subsidies offer a means to coverage for individuals with income at 100%-400% of the federal poverty level who do not qualify for Medicaid. OBJECTIVE: The objective of this study was to identify the effects of the ACA's non-Medicaid provisions on women's postpartum insurance coverage and depressive symptoms in nonexpansion states with low parental Medicaid thresholds. PARTICIPANTS: Women with incomes at 100%-400% of the federal poverty level who had prenatal insurance and completed the Pregnancy Risk Assessment Monitoring System (2012-2015). SETTING: Five non-Medicaid expansion states with Medicaid parental eligibility thresholds below the federal poverty level. DESIGN: Interrupted time-series analyses were conducted to examine changes between pre-ACA (January 2012-November 2013) and post-ACA (December 2013-December 2015) trends for self-reported loss of postpartum insurance and symptoms of postpartum depression. RESULTS: The sample included 9,472 women. Results showed significant post-ACA improvements where the: (1) trend for loss of postpartum insurance reversed (change of -0.26 percentage points per month, P=0.047) and (2) level of postpartum depressive symptoms decreased (change of -3.5 percentage points, P=0.042). CONCLUSIONS: In these 5 states, the ACA's non-Medicaid provisions were associated with large increases in retention of postpartum insurance and reductions in postpartum depressive symptoms, although depressive symptoms findings are sensitive to model specification.

Diabetes Prevention and Care Capacity at Urban Indian Health Organizations.

American Indian and Alaska Native (AI/AN) people suffer a disproportionate burden of diabetes and cardiovascular disease. Urban Indian Health Organizations (UIHOs) are an important source of diabetes services for urban AI/AN people. Two evidence-based interventions-diabetes prevention (DP) and healthy heart (HH)-have been implemented and evaluated primarily in rural, reservation settings. This work examines the capacity, challenges and strengths of UIHOs in implementing diabetes programs. Methods: We applied an original survey, supplemented with publicly-available data, to assess eight organizational capacity domains, strengths and challenges of UIHOs with respect to diabetes prevention and care. We summarized and compared (Fisher's and Kruskal-Wallis exact tests) items in each organizational capacity domain for DP and HH implementers vs. non-implementers and conducted a thematic analysis of strengths and challenges. Results: Of the 33 UIHOs providing services in 2017, individuals from 30 sites (91% of UIHOs) replied to the survey. Eight UIHOs (27%) had participated in either DP (n = 6) or HH (n = 2). Implementers reported having more staff than non-implementers (117.0 vs. 53.5; p = 0.02). Implementers had larger budgets, ~$10 million of total revenue compared to $2.5 million for non-implementers (p = 0.01). UIHO strengths included: physical infrastructure, dedicated leadership and staff, and community relationships. Areas to strengthen included: staff training and retention, ensuring sufficient and consistent funding, and data infrastructure. Conclusions: Strengthening UIHOs across organizational capacity domains will be important for implementing evidence-based diabetes interventions, increasing their uptake, and sustaining these interventions for AI/AN people living in urban areas of the U.S.

Development of a Health Information Technology Tool for Behavior Change to Address Obesity and Prevent Chronic Disease Among Adolescents: Designing for Dissemination and Sustainment Using the ORBIT Model.

Health information technology (HIT) has not been broadly adopted for use in outpatient healthcare settings to effectively address obesity in youth, especially among disadvantaged populations that face greater barriers to good health. A well-designed HIT tool can deliver behavior change recommendations and provide community resources to address this gap, and the Obesity-Related Behavioral Intervention Trials (ORBIT) model can guide its development and refinement. This article reports the application of the ORBIT model to (1) describe the characteristics and design of a novel HIT tool (the PREVENT tool) using behavioral theory, (2) illustrate the use of stakeholder-centered "designing for dissemination and sustainability" principles, and (3) discuss the practical implications and directions for future research. Two types of stakeholder engagement (customer discovery and user testing) were conducted with end users (outpatient healthcare teams). Customer discovery interviews (n = 20) informed PREVENT tool components and intervention targets by identifying (1) what healthcare teams (e.g., physicians, dietitians) identified as their most important "jobs to be done" in helping adolescents who are overweight/obese adopt healthy behaviors, (2) their most critical "pains" and "gains" related to overweight/obesity treatment, and (3) how they define success compared to competing alternatives. Interviews revealed the need for a tool to help healthcare teams efficiently deliver tailored, evidence-based behavior change recommendations, motivate patients, and follow-up with patients within the constraints of clinic schedules and workflows. The PREVENT tool was developed to meet these needs. It facilitates prevention discussions, delivers tailored, evidence-based recommendations for physical activity and food intake, includes an interactive map of community resources to support behavior change, and automates patient follow-up. Based on Self-Determination Theory, the PREVENT tool engages the patient to encourage competence and autonomy to motivate behavior change. The use of this intentional, user-centered design process should increase the likelihood of the intended outcomes (e.g., behavior change, weight stabilization/loss) and ultimately increase uptake, implementation success, and long-term results. After initial tool development, user-testing interviews (n = 13) were conducted using a think-aloud protocol that provided insight into users' (i.e., healthcare teams) cognitive processes, attitudes, and challenges when using the tool. Overall, the PREVENT tool was perceived to be useful, well-organized, and visually appealing.

Fostering innovation at Academic Medical Centers: The Case of University of Colorado Anschutz Medical Campus.

Commercializing biomedical discoveries is a challenging process for many reasons. However, Academic Medical Centers (AMC) that have teaching, patient care, research, and service engrained in their mission are well poised to host these discoveries. These academic discoveries can lead to improvement in patient health and economic development if supported to cross the "valley of death" through institutional assistance, by providing guidance, gap funding and product development expertise. Colorado has a vibrant local startup ecosystem, state support for commercialization and entrepreneurship as well as critical mass of product development expertise. University of Colorado Anschutz Medical Campus, as a major AMC, is an engine for growth for the region. This article discusses innovation efforts at the University of Colorado Anschutz Medical Campus as a case study, which is built around two major efforts: the CCTSI and CU Innovations. I-Corps at CCTSI and the SPARK|REACH program of CU Innovations have been instrumental in fostering innovation, commercialization, and entrepreneurship on the campus.

Pragmatic applications of implementation science frameworks to regulatory science: an assessment of FDA Risk Evaluation and Mitigation Strategies (REMS) (2014-2018).

BACKGROUND: A Risk Evaluation and Mitigation Strategy (REMS) is a drug safety program for certain medications with serious safety concerns required by the U.S. Food and Drug Administration (FDA) of manufacturers to implement to help ensure the benefits of the medication outweigh its risks. FDA is encouraging "the research community to develop novel methods for assessing REMS," conveying the unmet need for a standardized evaluation method of these regulatory-mandated healthcare programs. The objective of this research is to evaluate FDA REMS assessment plans using established implementation science frameworks and identify opportunities for strengthening REMS evaluation. METHODS: A content analysis was conducted of publicly available assessment plans for all REMS programs (N = 23) approved 1/1/2014-12/31/2018 for new drug applications (NDAs) and biologics license applications (BLAs) requiring FDA-mandated Elements to Assure Safe Use (ETASU). Blinded reviewers critically appraised REMS assessment measures (n = 674) using three established implementation science frameworks: RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance); PRECEDE-PROCEED (Predisposing, Reinforcing, and Enabling Constructs in Educational/Environmental Diagnosis and Evaluation - Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development); and CFIR (Consolidated Framework for Implementation Research). Framework constructs were mapped to REMS Assessment categories as defined by FDA Guidance for Industry to evaluate congruence. RESULTS: REMS assessment measures demonstrated strong congruence (> 90% mapping rate) with the evaluative constructs of RE-AIM, PRECEDE-PROCEED, and CFIR. Application of the frameworks revealed that REMS assessment measures heavily emphasize implementation and operations, focus less on health outcomes, and do not evaluate program context and design assumptions. CONCLUSIONS: Implementation science frameworks have utility for evaluating FDA-mandated drug safety programs including the selection of primary measures to determine whether REMS goals are being met and of secondary measures to evaluate contextual factors affecting REMS effectiveness in varying organizational settings.